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Preface 


This  research  was  an  effort  to  obtain  information 
about  the  frequency  of  occurrence  of  radar  system  write-ups 
on  the  F-16  aircraft.  It  was  undertaken  in  conjunction 
with  a  study  by  the  Air  Force  Human  Resources  Laboratory, 
Wright-Patterson  AFB,  Ohio  and  Westinghouse  Electric 
Corporation.  The  results  of  this  research  will  hopefully 
assist  the  Air  Force  in  determining  areas  where  the 
maintenance  decision  process  can  be  improved,  in  the  long 
run  saving  money  and  increasing  mission  capability. 

The  author  wishes  to  thank  Mr.  Russell  M.  Genet,  Air 
Force  Human  Resources  Laboratory,  Wright-Patterson  APB, 
Ohio  for  his  guidance  duriag  This  research,  and  Mr.  Leroy 
N.  Russell,  Dynamics  Research* Corporation ,  Dayton,  Ohio  for 
his  enthusiastic  support  and  assistance  in  obtaining  the 
required  data. 
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Abstract 


One  hundred  and  eight  aircraft  were  randomly  selected 
from  three  USAF  F-16  bases  and  examined^ over  the  time 


period  1  Dec  81  to  15  Aug  32.  >These  aircraft  included  63 
single-seat  F-16As  and  45  two-seat  F-16Bs  and  encompassed 
3,525  sorties  and  748  radar  system  write-ups.  Programs 
supported  by  the  Statistical  Package  for  the  Social 
Sciences  (SPSS)  were  run  on  the  data.  Of  the  748  discrep¬ 
ancies,  over  one-third  of  them  occurred  within  three 
sorties  of  each  other  and  half  within  six  sorties.  Sixteen 
percent  of  all  aircraft  which  had  a  discrepancy  within 
three  sorties  had  another  write-up  within  the  next  three 
sorties.  Designated  repeat/recurring  write-ups  represented 
one-third  of  all  the  instances  in  which  the  write-up 
separation  Interval  was  thre'e  sorties  or  less.  This  is  an 
indication  that  maintenance  is  unable  to  correct  equipment 
failures  as  they  occur,  roost  likely  because  the  false  alarm 
rate  is  too  high  and  maintenance  is  unable  to  duplicate  the 
error  conditions  on  the  ground  for  correct  error  diagnosis.r 
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I.  INTRODUCTION 

Recent  advances  in  technology  and  Air  Force  require¬ 
ments  have  given  birth  to  a  new  generation  of  aircraft 
avionics  systems.  These  new  systems  provide  crew  members 
with  more  information,  more  options  and  more  capability 
than  ever  before.  This  translates  into  a  higher  sortie 
effectiveness  (efficiency)  and  reduced  workloads  for  the 
crew  member  accomplishing  the  same  tasks.  In  the  case  of 
fighter  aircraft,  this  also  means  an  increase  in  the  sur¬ 
vivability  rates  during  enemy  engagements  and  higher  accu¬ 
racy  in  ordinance  delivery. 

Background 

The  Air  Force  currently 4)as  in  its  inventory  several 
aircraft  which  use  this  new  technology,  including  the  C-3A, 
F-15  and  F-16.  Although  these  new  systems  have  brought  the 
desired  improvements  in  capability  and  performance,  they 
have  also  brought  additional  maintenance  problems.  Besides 
being  higher  in  cost,  they  are  extremely  complicated,  re¬ 
quiring  complex  maintenance  equipment.  It  is  not  feasible 
to  use  highly  qualified  technicians  for  routine  maintenance 
and  repair  because  the  Air  Force's  supply  of  these  individ¬ 
uals  is  limited,  primarily  due  to  economic  and  manning  re¬ 
strictions.  Therefore,  the  maintenance  philosophy  has  been 
altered  from  one  of  "smart  man,  dumb  machine"  to  "smart 
machine,  dumb  man".  There  have  been  several  advances  in 
technology  which  make  this  philosophy  possible. 
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Specifically  self-test  and  built-in-test  functions  and 
automated  testing  procedures  (Ref  4:1). 

The  self-test  (ST)  and  built-in-test  (BIT)  functions 
provide  a  means  of  having  an  avionics  unit  test  itself 
while  it  is  operating  in  its  normal  environment.  Self-test 
has  been  defined  as  a  "continuous,  noninterruptive  fault 
detection  function  that  is  mode  oriented",  also  referred  to 
as  fault  detection  (FO) .  Built-in-test  has  been  defined  as 
a  "hierarchical  group  of  interruptive  tests  that  detect  and 
isolate  failures  to  a  single  line  replaceable  unit  (LR'J)”, 
also  called  fault  isolation  (PI)  (Ref  2:63,67). 

Automated  testing,  as  referred  to  in  this  research,  is 
the  computerized  testing  of  certain  avionics  units  in  a 
maintenance  shop  by  relatively:  .low  skill  level  technicians. 
They  communicate  information  'to  a  computer-controlled  test 
station  and  in  return  receive  test  results  and  instruc¬ 
tions.  This  test  equipment  is  referred  to  as  Automatic 
Test  Equipment  (ATE),  and  is  usually  located  in  the 
Avionics  Intermediate  Shop  (AIS)  where  Intermediate-Level 
(I-Level)  maintenance  is  performed. 

There  are  several  negative  aspects  to  self-test, 
built-in-test  and  automated  testing.  Using  current  tech¬ 
nology,  it  is  not  possible  to  achieve  the  extremely  high 
probabilities  of  fault  detection  and  fault  isolation 
desired  without  acccepting  a  high  false  alarm  rate. 
Measuring  the  actual  number  of  false  alarms  is  difficult 
because  the  measurement  is  clouded  by  the  fact  that  actual 
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failures  can  masquerade  as  false  alarms  (intermittent 
faults)  which  are  predominantly  the  result  of  BIT  specif!* 
cations  and  designs  being  tailored  to  an  ideal  (noise*£ree) 
world  (Ref  6:111,34-46). 

High  false  alarm  rates  significantly  affect  the  main¬ 
tenance  process.  The  technicians  learn  to  live  with  them, 
often  ignoring  a  failure  condition  even  if  it  is  (unknow¬ 
ingly)  correct.  A  study  of  nine  different  Air  Force 
systems  (Ref  2:7,15)  indicate  that  the  unnecessary  removal 
rate  of  LRUs  is  on  the  order  of  49%  and  has  been  found  as 
high  as  89%.  A  false  alarm  may  indicate  a  true  fault  that 
does  not  require  an  immediate  correction.  It  may  also 
indicate  a  degradation  in  system  performance  or  capability, 
particularly  if  the  arrival  periods  of  the  false  alarms 
(write-ups)  are  decreasin'g.  In  many  cases,  invalid 
indications  may  be  corrected  by  simply  resetting  the  BIT 
threshold.  However,  for  a  significant  reduction  in  BIT 
detection  errors,  testa  that  are  letting  too  many  bad  units 
go  undetected  need  to  be  tightened  up  and  tests  which  are 
identifying  too  many  good  units  as  faulty  need  to  be 
loosened  up,  a  trade-off  between  the  classic  Type  I  and 
Type  II  errors.  (Ref  8:15  and  Figure  1). 

BIT  usefulness,  the  percentage  of  field  problems 
resolved  by  using  BIT,  is  significantly  degraded  by  the 
presence  of  false  alarms.  If  BIT  indicates  a  momentary 
slgnsl  excursion  outside  of  the  test  limits,  the  operator 
can  be  reasonably  confident  that  the  signal  did  indeed 
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Diagnosed 

Condition 

Good 

Bad 

Actual 

Good 

Correct 

Type  I 

Condition 

Bad 

Type  II 

Correct 

Type  I  !  Reject  H(0)  when  it  is  true 
Type  II:  Accept  H(0)  when  it  is  false 

Fig  1.  Error  Diagnostics  Chart 


exceed  the  specified  limit.  But  more  often  than  not,  such 
anomalous  performance  is  not  a  manifestation  of  a  fault  and 
it  is  a  mistake  to  take  maintenance  action  based  solely  on 
the  BIT  indication  (Ref  6:vii) ,  When  an  Organizational- 
Level  (0-Level)  technician  tceubleshoots  a  system  at  the 
aircraft  and  fails  to  find  the  discrepancy  which  was  writ¬ 
ten  up,  the  result  is  a  Cannot  Duplicate  (CND)  report. 
Should  the  decision  be  made  to  remove  a  LRU  and  send  it  to 
the  Avionics  Intermediate  Shop,  if  the  Automatic  Test 
Equipment  fails  to  find  anything  wrong  with  the  unit  then  a 
Re-Test  Okay  (RTOK)  has  occurred. 

Currently,  CND  rates  average  in  excess  of  40%,  with 
RTOK  rates  around  30%  (Ref  1: 1,2:7).  These  are  averages 
throughout  military  and  commercial  aviation.  Several 
problems  are  associated  with  these  high  rates.  The  most 
significant  consequence  of  high  CND  and  RTOK  rates  is  that 
they  reduce  confidence  in  BIT  as  a  troubleshooting  tool, 
which  causes  BIT  to  be  ignored  at  times  even  if  it  is 
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correct.  In  addition,  they  result  in:  1)  increased  main¬ 
tenance  costs  and  rates,  2)  overcrowded  I-Level  maintenance 
shops,  3)  delays  in  aircraft  turnaround,  4)  inefficient  use 
of  high  skill  level  technicians  and  5)  reduced  mission 
capability. 

As  a  result  of  these  statistics  and  the  Air  Force's 
own  experience  with  the  new  systems,  the  Air  Force  Human 
Resources  Laboratory  (AFHRL)  is  conducting  a  pilot  study  to 
"develop  and  test  a  methodology  to  identify  the  causes  of 
diagnostic  errors  in  the  maintenance  of  avionics  equipment, 
quantify  their  relative  contributions,  and  develop  correc¬ 
tive  actions"  (Ref  1:1).  Westinghouse  Electric  Corporation 
(WEC)  has  been  contracted  to  provide  an  Avionics  Diagnos¬ 
tics  Pilot  Study  Plan  in  purs-uit  of  this  objective.  It  was 
desired  that  the  system_to.  be  investigated  be  a  military 
system  utilizing  ST/BIT,  with  at  least  two  years  of  opera¬ 
tional  service.  (Past  experience  indicates  that  it 
requires  two  to  three  years  of  operational  use  to  tailor 
the  BIT  in  a  new  system  to  an  operational  environment  and 
to  use  it  effectively  (Ref  2 :S-3 , 6 , 44 ) ) .  Additionaly,  a 
system  in  use  at  several  bases  was  desired  to  provide  a 
solid  data  base. 

The  F-16  was  chosen  for  the  study,  with  the  AN/APG-66 
Fire  Control  Radar  the  system  of  primary  interest.  The 
radar  system  is  the  most  critical  and  complex  part  of  the 
aircraft  avionics  package  and  employs  both  the  ST  and  BIT 
diagnostic  report  capabilities  which  require  maintenance 
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decisions  on  the  part  o£  maintenance  personnel.  Westing- 
house  further  recommended  the  Low  Power  Radio  Frequency 
(LPRF)  LRU  for  specific  analysis  because  it  is  the  most 
complex  LRU  in  the  radar  system  and  has  experienced  high 
CND  and  RTOK  rates.  Previous  studies  (Ref  2:27)  indicate 
that  the  overall  F-16  RTOK  rate  is  25.8%,  the  CND  rate  is 
45.6%,  the  fault  detection  rate  (attributable  to  Self-Test) 
is  49%  and  the  fault  isolation  rate  (attributable  to  Built- 
In-Test)  is  59%.  The  contracted  specification  for  the  fire 
control  radar  system  was  detection  and  Isolation  to 
specific  LRUs  for  95%  of  all  radar  malfunctions,  with  a 
false  alarm  rate  less  than  1%  (Ref  2:46).  As  the  Air  Force 
views  the  performance  statistics,  the  system  has  not  met 
its  contracted  specification^  . 

In  order  to  meet  the  goals  of  the  pilot  study,  AFHRL 
and  WEC  are  developing  a  model  of  the  maintenance  process 
which  will  be  used  to:  1)  determine  how  unnecessary 
maintenance  affects  aircraft  availability  and  support 
costs,  2)  identify  sensitive  decision  points  in  the  mainte¬ 
nance  process  and  3)  evaluate  suggested  improvements  (Ref 
4:1).  To  assist  in  gathering  the  necessary  information, 
AFHRL  sent  a  team  of  investigators  to  macDill  AFB,  Florida, 
where  the  maintenance  process  was  observed  first-hand. 

Initial  Study 

The  initial  area  of  study  for  this  thesis  was  the 
effect  of  the  "pilot  squawk”  (maintenance  write-up)  on  the 
maintenance  decision  process.  After  several  days  of 
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interviewing  F-16  pilots  and  instructors  and  examining  the 
specific  maintenance  process  at  MacDill  AFB,  it  was  deter¬ 
mined  that  although  the  pilot  is  the  only  person  to  observe 
the  performance  of  the  equipment  in  the  actual  conditions 
it  is  supposed  to  operate  under,  once  a  minimum  amount  of 
information  is  provided  to  maintenance  debriefers  further 
information  from  the  pilot  generally  does  not  affect  the 
maintenance  process.  Essentially,  the  pilot  becomes  a 
binary  flag  input,  indicating  malfunction  or  no 
malfunction. 

Results  of  the  pilot  interviews  verified  certain 
previous  assumptions  about  the  pilot  input.  Hard  failures, 
multiple  intermittent  failures,  noticeable  degradations  of 
performance  and  maintenance  jfai lure  codes  generated  by  the 
self-test  and  built- in-test'  functions  are  written  up. 
Occasional  intermittents  generally  are  not  written  up 
unless  supportive  information  (computer  diagnostic  error 
codes,  external  observations)  is  present.  Maintenance  then 
gets,  as  the  pilot  sees  it,  only  write-ups  on  known 
failures.  The  amount  and  type  of  information  conveyed  to 
maintenance  during  the  debriefing  is  highly  dependent  on 
the  individual  pilot,  the  debriefer  and  the  type  of 
malfunction.  As  a  result,  it  was  not  possible  to  obtain 
information  which  would  provide  a  solid  data  base  for 
analysis  and,  based  upon  the  information  received  during 
the  interviews,  this  particular  course  of  research  was 
terminated. 
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The  technician  who  goes  to  the  aircraft  to  see  if  he 
can  duplicate  the  fault  indication  often  knows  only  that  a 
particular  malfunction  code  was  reported  or  that  a  certain 
symptom  occurred.  Actual  parameters  such  as  altitude,  air¬ 
speed,  g-loading  and  outside  air  temperature  are  often 
unknown  to  the  technician,  and  his  procedure  is  essentially 
independent  of  the  amount  of  information  provided.  When  he 
runs  the  aiT  function  on  the  ground,  it  is  run  under  condi¬ 
tions  very  different  than  those  actually  experienced  in 
flight.  The  environmental  effect  on  decision  error  rates 
is  significant  although  not  as  significant  as  maintenance 
decision  errors  and  false  alarms  (Ref  3:4). 

If  the  results  of  the  BIT  agree  with  the  discrepancy 
reported,  the  appropriate  acirien  indicated  by  the  Technical 
Order  is  taken.  If  maintena^nce  can  be  delayed  until  after 
the  last  sortie  of  the  day  without  affecting  the  remaining 
mission(s),  then  it  most  likely  will  be.  Otherwise,  if  LRU 
removal  is  directed,  the  unit  is  sent  to  the  AIS.  If  a 
spare  is  available,  it  is  replaced  as  soon  as  practical. 
If  there  is  no  spare  unit,  then  maintenance  has  two  alter¬ 
natives:  1)  cannibalize  a  good  unit  from  another  aircraft 
or  2)  place  the  aircraft  in  a  restricted  status  which  will 
most  likely  remove  the  aircraft  from  the  flying  schedule. 

Specific  action  may  also  be  dictated  by  the  previous 
history  of  the  aircraft.  If  the  discrepancy  is  occurring 
for  the  only  time  within  the  last  three  sorties  then  normal 
maintenance  procedure  is  followed.  If  aircraft  history 
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records  indicate  that  the  same  discrepancy  was  reported 
within  the  last  three  sorties  then  the  write-up  is  anno¬ 
tated  with  an  additional  maintenance  repeat/recurring  code 
of  'C  or  'R',  If  the  previous  write-up  does  not  show  a 
repeat/recurring  code  then  this  is  the  second  occurrence 
and  normal  maintenance  procedures  are  followed.  If,  how¬ 
ever,  a  'C  or  'R*  code  exists  on  the  preceding  discrepancy 
then  this  write-up  is  the  third  occurrence  and  policies 
dictate  that  positive  maintenance  be  performed.  This  is  an 
attempt  to  preclude  the  intermittent  from  continually 
arising  and  degrading  system  performance.  If  flight  line 
maintenance  technicians  fail  to  locate  a  fault  at  the 
aircraft,  the  LRU  which  is  deemed  by  these  technicians  to 
be  the  most  likely  cause  o£  the  problem  is  removed  and  sent 
to  the  I-Level  maintenance.slxop  with  the  hope  that  this 
will  correct  the  malfunction. 
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II.  STATEMENT  OF  THE  PROBLEM 


The  high  CUD,  RTOK  and  false  alarm  rates  of  the  P>16 
radar  system  have  a  significant  impact  on  the  Air  Force 
mission  and  capability.  In  conjunction  with  the  Avionics 
Diagnostic  Pilot  Study  Plan,  it  was  believed  that  an 
analysis  of  the  radar  discrepancies  would  be  beneficial. 
Specifically,  this  would  include  a  study  of  the  rates  of 
occurrence  of  radar  system  write-ups  in  general,  the 
characteristics  of  their  occurrences  and  the  character¬ 
istics  of  the  repeat/recurring  write-ups.  Once  the  data 
were  obtained  and  analyzed,  a  simple  model  portraying  the 
probability  of  a  radar  discrepancy  on  a  given  sortie  could 
be  constructed.  The  remainder  of  this  thesis  is  the  report 
on  this  effort  at  analysis-'of  the  P-16  radar  system  dis¬ 
crepancies.  * 


Approach 

The  F-16  maintenance  records  are  maintained  on  a 
computer  system  known  as  the  Centralized  Data  System  (CDS) 
by  Dynamics  Research  Corporation  (DRC) ,  a  civilian  contrac¬ 
tor.  This  system  is  a  significant  improvement  over 
previous  data  storage  methods  in  that  it  provides  access  to 
specific  information  within  24  hours  from  the  time  that  it 
is  entered  into  the  system.  In  addition,  users  have  access 
to  the  P-16  maintenance  records  of  all  USAF  P-16  bases 
which  are  presently  connected  to  the  system. 

Data  are  organized  in  a  complicated  structure  in  the 
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CDS,  somewhat  similar  to  a  multiple  linked- record  format 
but  with  many  inter-connections.  In  order  to  simplify  data 
access  for  the  everyday  user,  standard  queries  have  been 
created  by  DRC.  A  standard  query  initially  modified  by  Mr. 
L.  Russell  of  DRC  and  provided  by  him  was  further  modified 
in  order  to  obtain  the  required  information  for  this  study. 
This  particular  program  provides  a  listing  of  all  sorties 
flown  and  the  associated,  if  any,  maintenance  malfunction 
codes.  (Appendix  A). 

To  provide  a  data  base  for  the  analysis,  records  of 
108  aircraft  which  were  selected  at  random  from  3  bases 
were  obtained,  which  encompassed  8,525  sorties.  This  popu¬ 
lation  included  both  the  single-seat  F-16A  and  the  two-seat 
F-16B,  the  B  model  being  the.4:rainer  version.  The  actual 
distribution  of  aircraft,  fs  shown  in  Appendix  B.  These 
aircraft  represent  approximately  one-third  of  the  number  of 
aircraft  stationed  at  these  three  bases  during  the  time 
period  1  December  1981  to  15  August  1982,  the  time  frame 
for  which  data  were  obtained. 

The  information  from  the  Centralized  Data  System  was 
then  recoded  into  a  format  suitable  for  processing  by 
various  programs  supported  by  the  statistical  Package  for 
the  Social  Sciences  (SPSS).  A  separate  program  was  written 
to  analyze  the  arrival  characteristics  of  the  data  and  of 
the  repeat/recurring  discrepancies. 

Of  particular  interest  to  the  AFHRL  for  their  pilot 
study  is  the  number  of  sorties  separating  multiple 
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consecutive  write-ups.  The  original  data  represent  the 
sorties  separating  two  write-ups.  Sorties  separating  three 
write-ups  can  be  analyzed  using  the  following  approach. 
Determine  the  first  (reference)  write-up.  Identify  i  as 
the  number  of  sorties  separating  the  first  write-up  from 
the  next  (second)  write-up  and  j  as  the  number  of  sorties 
separating  the  second  write-up  from  the  third.  An  analysis 
can  then  be  made  on  the  ordered  pairs  (i^j).  Extending 
this  process  to  one  more  write-up,  an  analysis  can  also  be 
made  of  the  ordered  triples  (irjfl()f  where  k  is  the 
number  of  sorties  separating  the  third  write-up  from  the 
fourth.  These  results  were  then  used  to  create  a  simple 
model  for  use  by  the  AFHRL  in  their  study. 
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III.  RESULTS 


The  data  maintained  by  the  Centralized  Data  System 


are,  in  certain  time  frames,  incomplete  and  occasionally 


inconsistent  across  data  records.  For  this  research,  the 


data  range  was  restricted  to  the  nine  and  one-half  month 


period  beginning  1  December  1931,  primarily  because  this 


was  the  break  point  for  current  on-line  data  and  data  which 


needed  to  be  accessed  separately  through  historical  re¬ 


cords.  In  addition  to  sporadic  omissions  of  data,  there 


was  a  one  and  one-half  month  break  in  the  records  for  air¬ 


craft  from  one  base  which  could  not  be  recovered  using  any 


of  the  other  data  records  available. 


Pertinent  data  were  used  from  the  first  occurrence  of 


a  radar  system  write-up^  to  the  last  occurrence  in  the 


specified  time  frame.  The  number  of  sorties  from  1  Dec  81 


to  the  first  write-up,  and  from  the  last  write-up  to  15  Aug 


82,  were  ignored  because  the  actual  number  of  sorties 


separating  the  last  and  first  discrepancies  outside  the 


time  frame  was  unknown. 


The  8,525  sorties  included  in  this  study  resulted  in 


748  occurrences  of  radar  write-ups.  The  SPSS  programs 


showed  that  the  average  separation  between  write-ups  at  all 


three  bases  was  11.4  sorties,  with  a  minimum  of  1  (i.e.. 


occurrence  on  the  next  sortie)  and  a  maximum  of  135.  The 


most  frequently  observed  separation  (the  mode)  was  1, 


occurring  14.8%  of  the  time. 


i-v:-  :.r:  < 
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Probably  the  most  meaningful  statistic,  however,  is 
the  median,  that  number  which  has  50%  of  the  write-ups  on 
either  side  of  it.  The  median  for  the  overall  population 
was  6.4.  Approximately  35%  of  all  write-ups  occurred 
within  3  sorties  of  another  discrepancy,  50%  within  6 
sorties  and  66%  within  10  sorties.  For  a  complete  break¬ 
down  by  base  and  model,  see  Appendices  C  and  D. 

The  distribution  of  the  probabilities  of  having  i 
sortie  separations  between  write-ups  initially  appears  to 
be  an  exponentially  decreasing  function.  Further  analysis 
using  linear  regression  techniques  reveals  that  this 
particular  distribution  is  approximated  very  closely  by  a 
function  of  the  form 


P(i) 


-0.im  +  12.761/i  +  3.371 


where  i  is  the  number  of  sorties  until  the  next  write-up 
and  P(i)  is  the  probability  of  that  occurrence,  in 
percent.  (The  R-Square  correlation  value  for  this  approx¬ 
imation  is  0.9658).  A  graph  of  the  actual  distribution  and 
this  approximation  are  shown  in  Figure  2. 

The  median  number  of  sorties  separating  radar  write¬ 
ups  at  Nellis  is  4.8,  significantly  less  than  the  medians 
for  Hill  (7.1)  and  MacDill  (6.4).  This  might  be  explained 
by  the  fact  that  Nellis'  operation  is  considerably  newer 
than  either  of  the  others  and,  as  a  consequence,  the  expe¬ 
rience  level  of  their  maintenance  technicians  may  be  lower. 
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Incorrect  maintenance  decisions  might  then  occur  more 
frequently. 

Also  of  significance  is  that,  in  general,  the  A  model 
discrepancies  occurred  more  often  than  those  for  the  B 
model  (every  5.9  as  opposed  to  7.2).  This  was  also  true 
when  comparing  the  models  at  individual  bases,  except  at 
Nellis.  The  difference  could  be  explained  by  the  follow¬ 
ing.  For  most  training  sorties,  pilots  usually  fly  in  the 
A  model.  The  B  model  is  used  primarily  when  the  training 
syllabus  requires  an  instructor  to  fly  in  the  same  air¬ 
craft.  This  occurs  normally  when  a  pilot  is  being  intro¬ 
duced  to  a  new  phase  of  flying  (e.g.,  transition,  air-to- 
air,  air-to-ground).  Once  the  pilot  has  obtained  a  minimum 
level  of  proficiency,  furthec- training  is  normally  obtained 
using  the  A  model.  Also,  tot'al  system  utilization  in  the  B 
model  does  not  occur  as  often  as  as  it  does  in  the  A  model 
because  of  the  specialization  of  the  B  model  sorties.  As  a 
result,  it  is  possible  for  discrepancies  to  occur  more 
often  in  the  A  model  than  in  the  B  model.  It  is  worth 
mentioning  that  for  scheduling  and  training  flexibility  a  B 
model  aircraft  is  often  flown  single-seat  when  an  A  model 
would  suffice.  This  explanation  then  is  offered  purely  as 
a  supposition. 

Analysis  of  the  ordered  pairs  (i,j),  as  previously 
described,  yields  interesting  results.  Of  the  S40  total 
sortie  pairs  involved,  107  which  had  a  radar  discrepancy 
within  three  sorties  of  a  reference  write-up  had  another 
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within  the  next  three,  or  16.7%.  Twenty-seven  percent 
which  had  a  write~up  within  five  sortiea  had  one  again 
within  the  next  five.  Figure  3  is  a  partial  matrix 
extracted  from  the  complete  listing  in  Appendix  E. 

j,  (Following  Interval) 

i, 

(Initial 

Interval) 

i,j:  number  of  sorties  between  radar  write-ups 
Fig  3.  Distribution  of  Consecutive  write-ups 

There  were  51  occurrences  of  the  ordered  triples 
(irj#h),  where  each  successive  write-up  occurred  within 
three  sorties  of  the  previous  one,  out  of  the  541  total 
possible,  or  9.4%.  In  addition,  sixteen  percent  (87)  fell 
into  the  category  where  there  were  five  or  less  sorties 
separating  consecutive  write-ups. 

Results  of  the  distribution  of  i  alone  and  the 
ordered  pairs  (i,j)  can  be  combined  to  form  a  model.  (The 
ordered  triples  (i,j,k)  are  not  included  in  this  n»del 
because  the  model  would  then  lose  its  simplicity).  This 
model  provides  probabilities  for  how  many  sorties  will 
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occur  before  another  discrepancy  is  encountered,  given  a 
known  initial  separation,  for  inclusion  in  the  AFHRL's 
maintenance  decision  diagnostic  model,  and  is  shown  in 
Figure  4. 

Analysis  of  the  repeat/recurring  coded  write-ups 
resulted  in  the  following  statistics.  Of  the  108  aircraft 
investigated,  39  had  a  total  of  79  write-ups  designated  as 
repeat/recurring.  Fourteen  of  these  discrepancies  occurred 
immediately  following  a  similarly  coded  write-up  (i.e., 
three  identical  complaints  in  a  row) .  This  involved  nine 
of  the  thirty-nine  aircraft.  The  79  occurrences  represent 
two-thirds  of  all  the  instances  (107)  where  a  discrepancy 
occurred  within  the  next  three  sorties.  The  fourteen 
immediately  repeated  write*^ps  represent  thirteen  percent 
of  the  total  number  of  (i,j)  'pairs  where  successive  write¬ 
ups  occurred  within  three  sorties  of  each  other. 
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Number (i)  Number (j) 

(Pdi )  i  (P{j)) 


18  (2.8)- 
21(3.3) 
10  (1.6)- 
8 (1.3) 
4 (0.6) 


15(2.3) 
16 (2.5) 
7(1.1) 

4  (0.6) 
3(0.5)- 


8(1.3) 
6 (0.9) 
6(0.9) 
6(0.9) 
4(0.6) 


8(1.3)- 

2(0.3) 

3(0.5) 

4(0.6) 

3(0.5) 


4(0.6) 
2(0.3) 
5(0.8) 
2(0.3) 
4 (0.6) 


Total  Possible:  748 


640 


i,j:  number  of  sorties  between  radar  write-ups 


Fig  4.  Model  of  Consecutive  Write-ups  (i^j) 
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IV.  CONCLUSIONS 


This  research  has  shown  that  a  significant  portion  of 
all  radar  discrepancies  occur  within  three  sorties  of  a 
previous  write-up.  This  would  be  a  result  of  inappropriate 
BIT  tests,  intermittents,  false  alarms  and  incorrect 
maintenance  decisions,  as  previously  discussed.  Equipment 
which  was  undergoing  a  gradual  component  failure  or  tended 
to  fail  only  under  certain  environmental  conditions  would 
greatly  increase  the  CND  rate  since  this  condition  would 
most  likely  be  extremely  difficult  to  isolate  during  ground 
testing.  Because  of  the  various  mission  profiles  which  are 
flown,  not  all  of  the  radar  system  capabilities  are  exer¬ 
cised  on  every  flight.  This  could  explain  the  difference 
between  the  A  model  and  B>4BO(fel  statistics  since  certain 
missions  are  typically  flovn  in  a  particular  model  of 
aircraft.  It  would  also  explain  why  write-ups  do  not 
always  occur  on  consecutive  sorties. 

The  analysis  of  both  total  occurrences  of  radar  write¬ 
ups  and  the  ordered  pair  occurrences  (i,j)  has  provided  the 
basis  for  a  simple  model.  The  model  indicates  that  there 
is  indeed  a  trend  in  the  occurrences  of  radar  system  write¬ 
ups.  The  trend  is  that  if  a  discrepancy  exists,  there  is  a 
good  chance  that  another  write-up  will  occur  in  the  next 
few  sorties,  and  if  another  occurs,  that  it  too  will  be 
within  the  next  few  sorties.  This  is  not  to  imply  that 
they  are  all  the  same  write-up;  however,  the  conclusion 
that  they  are  related  might  not  be  inappropriate.  The 
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causes  for  this  trend  are  likely  to  be  those  previously 
mentioned,  since  actual  equipment  failures  alone  of  the 
magnitude  necessary  to  produce  the  same  result  are  highly 
improbable . 

Write-ups  which  were  identified  as  repeat/recurring  in 
the  aircraft  records  provide  approximations  to  rates  of 
occurrence.  However,  actual  identification  of  a  discrep¬ 
ancy  as  repeat/recurring  is  dependent  upon  the  nature  of 
the  write-up,  the  history  information  available  during 
pilot  debrief  and  the  debriefer's  ability  to  accurately 
assess  the  complaint.  This  information  should  therefore  be 
used  only  as  a  guide. 

e 
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V.  Recommendations  for  Further  Study 

The  area  of  repeat/recurring  write-ups  is  one  which 
should  be  further  investigated.  It  was  not  covered  in 
depth  in  this  research  because  of  the  time  committment 
required  to  go  through  the  aircraft  historical  records,  in 
order  to  obtain  accurate  data,  the  actual  write-up  needs  to 
be  traced  from  its  inception  (post-flight  debrief)  to  its 
close-out.  Initial  study  in  this  area  indicates  that  the 
computer  records  are  often  incomplete,  providing  no  record 
of  actual  corrective  action  performed.  An  in-depth  tracing 
of  these  write-ups  would  provide  more  data  for  localizing 


write-ups  and 

faults 

to  particular  LRUs 

and  also 

provide 

an 

indication  of 

how 

maintenance  goes 

about 

correcting 

discrepancies 

when 

sp<M»-ifIc  repair 

information 

is 

unavailable . 

-  - 
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Appendix  A 

CDS  Data  Retrieval  Program 

The  following  program  is  a  second-generation  modi¬ 
fication  of  the  CDS  standard  query  labeled  "B-RPT".  It 
provides  a  chronological  listing  of  an  aircraft's  landing 
status  for  each  sortie  flown,  and  the  assigned  work-unit- 
code  and  an  abbreviated  comment  on  the  malfunctioning 
system  if  any  write-up  occurs  during  post-flight 
debriefing. 

The  input  parameters  as  modified  are  a  base  identi¬ 
fication  code,  a  specific  tail  number  or  the  word  "ALL”, 
and  a  model  identification  of  "A",  "B"  or  "C",  where  "C 
combines  all  A  and  B  models. ^The  date  range  can  be  changed 
by  modifying  the  statements  Irabeled  R0  and  Rl. 
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Appendix  A  (cont'd) 


OPTIONS  ARE  ALLOCATED  CORE  «  URGENCY  *  30,  TIME  « 
INVOKE  41380T/F16/YPEADF 
LI.  LET  B1  *  IBASE. 

L6.  LET  TAIL  «  #TAIL-NUMBER. 

L7.  LET  MODEL  »  IMODEL. 

IF  MODEL  «  "A" 

LET  MODI  ■  "AFA" 

THEN  LET  MOD2  «  "XPR" 

THEN  LET  MOD3  »  "AYL" 

THEN  LET  MOD4  *  "XYL" 

THEN  LET  MODS  «  "AYN" 

THEN  LET  MOD 6  »  "XYN" 

THEN  LET  MOD 7  »  "AYQ" 

THEN  LET  MOD8  »  "XYO" 

THEN  LET  MOD9  *  "AYS" 

THEN  LET  MOD10  =  "XYS" 

IF  MODEL  «  "a" 

LET  MODI  »  "ANF" 

THEN  LET  MOD2  •  "XNL" 

THEN  LET  MOD3  »  "AYM" 

THEN  LET  MOD4-^  "^YM" 

THEN  LET  MODS  "AYP" 

THEN  LET “MODS  »  "XYP" 

THEN  LET  MOD7  «  "AYR" 

THEN  LET  MODS  ■  "XYR" 

THEM  LET  MOD9  »  "AYT" 

THEN  LET  MOD10  »  "XYT" 

IF  Bl  »  "USAT" 

let  Bl  ■  "KRSM" 

THEN  LET  B6  «  "VLSB" 

THEN  LET  B2  ■  "NVZR" 

THEN  LET  B3  »  "PTFA" 

THEN  LET  BS  »  "RXMF" 

THEN  LET  B4  »  "FSPM". 

IF  Bl  «  "USAF" 

LET  Bl  ■  "KRSM" 

THEN  LET  B6  »  "VLSB" 

THEN  LET  B2  ■  "NVZR" 

THEN  LET  BS  ■  "RKMF". 

IF  Bl  ■  "NETH" 

LET  Bl  -  "NBBW" 

THEN  LET  B2  -  "CCCC" 

THEN  LET  B3  »  "DDDD". 


(cont'd) 
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Appendix  A  (cont'd) 


IP  81  *  "NORW" 

LET  B1  «  "UPSA" 

THEN  LET  82  »  "UPSB" 

THEN  LET  83  »  "FPFP". 

IF  81  =  "BELG" 

LET  81  «  "BBS2" 

THEN  LET  32  »  "MaOB* 

THEN  LET  83  «  "HHHH*. 

IF  81  *  "DENM" 

LET  81  «  "VTLV 

THEN  LET  82  «  "IIII" 

THEN  LET  83  «  "JJJJ". 

IF  81  »  "PACAF" 

LET  81  «  "MLWR" 

THEN  LET  82  «  "LXEE* 

THEN  LET  83  «  “AAAA". 

IF  81  -  "USAFE" 

LET  81  ■  "JWEC 

THEN  LET  82  «  •BBBB" 

THEN  LET  83  «  "KKEK". 

R0.  RETRIEVE  FLEET-BASE  FROM  YPE  WHERE 

(FLEET-DATE  ■  •8112"  OR  ■Ji201*  OR  "8202"  OR  ■8203*  OR 
"8204*  OR  •8205*  OR  •8«I6*  OR  •8207"  OR  *8208"  ) 

AND  (LOC-B-C  »  B1  OR  B2  OR  B3  OR  B4  OR  85  OR  86) 

WHEN  R0.  “ 

Rl.  RETRIEVE  M-BREC  FROM  YPE  WHERE 

(B-FLY-DT  BETWEEN  "811201"  AND  "821815"  ) 

AND  (B-LAND-ST  »  "1"  OR  "2"  OR  "3"  OR  "4"  OR  "  "  ) 

WHEN  Rl. 

IF  MODEL  ■  "C"  GO  TO  Jl. 

IF  B-SRD  NOT  ■  MODI  AND  MOD2  AND  MOD3  AND  MOD4  AND  MOD5 
AND  MODS  AND  MOD?  AND  MOD8  AND  NOD9  AND  MOD10  RETURN. 

Jl. 

IF  TAIL  »  "ALL"  GO  TO  J2. 

IF  B-TAILNR  MOT  «  TAIL  RETURN. 

J2. 

SI.  SORT  WITHIN  Rg  M-BREC  ON  B-TAILNR,  B-FLY-DT,  B-SORT-NUM, 
B-SORT-SEQ 
WHEN  SI. 

PRINT  ON  FILE  #REPORT  FOR  TEK  B-TAILNR,  B-FLY-DT,  B-SORT-NUM 
B-SORT-SEQ,  B-LAND-ST,  B-LAND-ST-WUC ,  B-LAND-ST-RMK, 
B-ACTUAL-TOFP. 

WHEN  Rg,  Rl  EMPTY 

PRINT  ON  FILE  IREPORT  FOR  TEK  "NO  DATA  FOUND  FOR 
CRITERIA  SPECIFIED" 


jl  ■■  i,  r-ij- 
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Appendix  B 

Distribution  of  Sample  Population 


Base 


Model 


Number  of 
Aircraft 


Number  of 
Sorties 


MacDill 

B 

* 

A 

Mellis 

B 

Total 

I“^’i 
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Appendix  C 

Single  Interval  Separation  Distributions 


Hill  AFB 


Humber  of  Occurrences  (%) 


Sortie 

Separation 


A 

Model 

B 

Model 

Combined 

18  (11.4) 

39  (12.1) 

19  (11.6) 

10  (6.3) 

29 (9.0) 

15(9.1) 

14 (8.9) 

29 (9.0) 

11(6.7) 

12(7.6) 

23(7.1) 

9(5.5) 

8(5.1) 

17(5.3) 

7(4.3) 

8(5.1) 

15(4.7) 

9(5.5) 

6(3.8) 

15(4.7) 

5(3.1) 

11 (7.0) 

16(5.0) 

5(3.1) 

7(4.4) 

12(3.7) 

7(4.3) 

6(3.8) 

13(4.0) 

4(2.4) 

2(1.3) 

6(1.9) 

2(1.2) 

1(0.6) 

3(0.9) 

4(2.5) 


3(1.8)  - 

2(1.3) 

5(1.6) 

4(2.4-)  / 

5(3.2) 

9(2.8) 

4(2.4) 

2(1.3) 

6(1.9) 

- 

4(2.5) 

4(1.2) 

2(1.2) 

3(1.9) 

5(1.6) 

1(0.6) 

3(1.9) 

4(1.2) 

2(1.2) 

3(1.9) 

5(1.6) 

1 (0.6) 

1(0.6) 

2(0.6} 

3(1.8) 

- 

3(0.9) 

2(1.2) 

2(1.3) 

4(1.2) 

1(0.6) 

1(0.6) 

2(0.6) 

- 

2(1.3) 

2(0.6) 

3(1.8) 

- 

3(0.9) 

2(2.4) 

1(0.6) 

3(0.9) 

1(0.6) 

1(0.6) 

2(0.6) 

2(1.2) 

7(4.4) 

9(2.7) 

4(2.4) 

1(0.6) 

5(1.5) 

8(4.9) 

11(7.0) 

19 (5.7) 

164(100) 


158 (100) 


332(100) 
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Appendix  C  (cont'd) 


MacDill  AFB 


Number  of  Occurrences  (%) 


Sortie 

Separation 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 


A 

Model 

B 

Model 

Combined 

35(19.6) 

12(11.9) 

47  (16.8) 

16(8.9) 

17  (16.8) 

33(11.8) 

12(6.7) 

7(6.9) 

19  (6.8) 

13(7.3) 

2(2.0) 

15(5.4) 

10(5.6) 

3(3.0) 

13(4.6) 

9(5.0) 

5(5.0) 

14 (5.0) 

8(4.5) 

5(5.0) 

13(4.6) 

4(2.2) 

2(2.0) 

6(2.1) 

8(4.5) 

2(2.0) 

10 (3.6) 

4(2.2) 

2(2.0) 

6(2.1) 

7(3.9) 

5(5.0) 

12(4.3) 

4(2.2) 

4(4.0) 

8 (2.9) 

2(1.1)  _  . 

2(2.0) 

4(1.4) 

5(2r«.) 

2(2.0) 

7(2.5) 

2(1.1)  - 

1(1.0) 

3(1.1) 

3(1.T) 

1(1.0) 

4(1.4) 

1(0.6) 

2(2.0) 

3(1.1) 

4(2.2) 

- 

4(1.4) 

4(2.2) 

2(2.0) 

6(2.1) 

2(1.1) 

4(4.0) 

6(2.1) 

2(1.1) 

1(1.0) 

3(1.1) 

2(1.1) 

— 

2(0.7) 

1(0.6) 

3(3.0) 

4(1.4) 

1(0.6) 

1(1.0) 

2(0.7) 

1(0.6) 

- 

1(9.4) 

- 

1(1.0) 

1(0.4) 

3(1.7) 

- 

3(1.1) 

1(0.6) 

1(1.0) 

2(0.7) 

6(3.4) 

2(2.0) 

8 (2.9) 

3(1.7) 

3(3.0) 

6(2.1) 

6(3.4) 

10 (9.9) 

16(5.7) 

179 (100) 


101  (100) 


280(100) 
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Appendix  C  (cont'd) 


Nellis  AFB 


Number  of  Occurrences  (%) 


Sortie 

A 

B 

Separation 

Model 

Model 

Combined 

1 

17(17.0) 

8(17.4) 

25(17.1) 

2 

17(17.0) 

7(15.2) 

24  (16.4) 

3 

9  (9.0) 

7(15.2) 

16  (11.0) 

4 

4(4.0) 

2(4.3) 

6(4.1) 

5 

6 (6.0) 

2(4.3) 

8(5.5) 

6 

2(2.0) 

3(6.5) 

5(3.4) 

7 

5(5.0) 

- 

5(3.4) 

8 

6(6.0) 

1(2.2) 

7(4.8) 

9 

5(5.0) 

3(6.5) 

8(5.5) 

10 

5(5.0) 

- 

5(3.4) 

11 

4(4.0) 

- 

4(2.7) 

12 

3(3.0) 

- 

3(2.1) 

13 

- 

3(6.5) 

3(2.1) 

14 

1(1.A)  ^ 

2(4.3) 

3(2.1) 

15 

4(4.0)  ' 

3(6.5) 

7(4.8) 

16 

3(3.0)  • 

- 

3(2.1) 

17 

1(1.0)  ■ 

1(2.2) 

2(1.4) 

18 

- 

1(2.2) 

1(0.7) 

19 

1(1.0) 

- 

1(0.7) 

20 

2(2.0) 

- 

2(1.4) 

21 

- 

1(2.2) 

1(0.7) 

22 

1(1.0) 

- 

1(0.7) 

23 

1(1.0) 

- 

1(0.7) 

24 

25 

26 

1(1.0) 

- 

1(0.7) 

27 

28 

29 

30 

31-35 

2(2.0) 

- 

2(1.4) 

36-40 

>40 

- 

2(4.3) 

2(1.4) 

Total 

100 (100) 

46 (100) 

146  (100) 

30 


L*--* 
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Appendix 

C  (cont'd) 

All 

Bases 

Number  of  Occurrences  (%) 

Sortie 

A 

8 

Separation 

Model  Combined 

52(11.7) 
36  (8.1) 
28  (6.3) 
25(5.6) 
18  (4.1) 
22(5.0) 
15(3.4) 
18  (4.1) 
16(3.6) 
15(3.4) 
9(2.0) 
4(8.9) 
ll(2r<^)  ' 
9(2.0)  - 
10(2. J)  • 
6(1.4)' 
4(0.9) 
7(1.6) 
5(1.1) 
4(0.9) 
2(0.5) 
6(1.4) 
3(0.7) 
2(0.5) 
2(0.5) 
3(0.7) 
5(1.1) 
1(0.2) 
1(0.2) 
10(2.3) 
7(1.6) 
14(3.2) 


34  (11.1) 
28 (9.2) 
16 (5.2) 
13(4.3) 
16 (5.2) 
11(3.6) 
14  (4.6) 
12(3.9) 

8  (2.6) 
7(2.3) 
5(1.6) 
9(3.0) 
6(2.0) 
6(2.0) 
6(2.0) 
5(1.6) 
5(1.6) 
5(1.6) 
7(2.3) 
5(1.6) 
1(0.3) 

5(1.6) 

2(0.7) 

2(0.7) 

1(0.3) 

1(0.3) 

1(0.3) 

1(0.3) 

9(3.0) 

3(1.0) 

23(7.5) 


111  (14.8) 
36  (11.5) 
64 (8.6) 
44 (5.9) 
38  (5.1) 
34 (4.5) 
33(4.4) 

29  {3i9) 

30  (4.0) 
24(3.2) 
22(2.9) 
14  (1.9) 
13(1.7) 
17(2.3) 
15(2.0) 
16 (2.1) 
11(1.5) 

9(1.2) 
12(1.6) 
12(1.6) 
9(1.2) 
3(0.4) 
6(0.8) 
8(1.1) 
4(0.5) 
4(0.5) 
4(0.5) 
6(0.8) 
2(0.3) 
2(0.3) 
19  (2.5) 
10(1.3) 
37(4.9) 


443(100)  305(100) 


748  (100) 
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Appendix  D 

Overall  Single  Interval  Separation  Statistics 


Std 


Base/Model 

Number 

Max 

Mode 

Mean 

Median 

Dev 

Hill/A 

164 

61 

1 

11.3 

5.5 

12.6 

Hill/B 

158 

135 

1 

13.5 

7.8 

17.4 

MacDill/A 

179 

110 

1 

.  10.9 

5.9 

14.2 

MacDill/B 

101 

102 

2 

14.5 

7.4 

18.2 

Nellis/A 

100 

34 

1  &  2 

7.3 

5.0 

7.0 

Nellis/B 

46 

57 

1 

8.7 

4.0 

12.2 

Hill/Both 

322 

135  ^ 

1 

12.4 

7.1 

15.2 

MacDill/Both 

280 

llT  - 

1 

12.2 

6.4 

15.8 

Nellis/Both 

146 

67  ‘ 

1 

7.7 

4.8 

9.0 

All/A 

443 

110 

1 

10.2 

5.9 

12.4 

All/B 

305 

135 

1 

13.1 

7.2 

17.0 

All/Both 


Number  of  Sorties,  i,  between  1st  and  2nd  Write-ups 
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Appendix  E 

Distribution  of  Write-ups  of  the  Ordered  Pairs  (i.j) 


(All  Bases) 


Number  of  Sorties,  j,  between  2nd  and  3rd  Write-ups 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 
31-35 
36-40 

>40 


18  21  10  3 

15  16  7  4 

8  6  6  6 


2  3 
2  5 
4  3 
2  - 
2  4 


5 


3  4 

4  3 
6  4 


3  4 
5  2 


2  2 
1  2 


6  7  8  9  10  11 


4  4  3  3  1  4 

5  1  3  4  2  1 

-24-1- 

1  2  3  3  -1 

13  3-12 

-212-- 
-12-13 
-  -  1  -  3  1 

2  2  111- 


-  1  2  1  - 
111-2---1 
1  -  -  1- 


-  -  1 


-  -  1 


---1---1 

2-1-1--- 


-11----1--2 

11--------- 

-11--1----- 

---------1- 

1---12213-1 

-2-11-1--1- 

723-11-113- 


-  1  -  - 


-  -  1  - 


Total  100  78  56  38  31  29  28  27  25  19  13  11  12  14  13 


Number  of  Sorties,  i,  between  1st  and  2nd  Write-ups 
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Appendix  E  (cont'd) 


Number  of  Sorties,  j,  between  2nd  and  3rd  Write-ups 


31  36  > 


Total  13  9  7  10  10  3  2  6  7  2  3  3  6  2  1  15  9  22 
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